1. Field of the Invention
The present invention is directed to a system and to a method for generating an image that contains superimposed or fused image data, having a first system for acquiring an image dataset of a subject and second system for registering a video image of the subject.
2. Description of the Prior Art
Given minimally invasive surgical interventions, for example in endoscopy or laparoscopy wherein medical instruments are introduced into the body of a life form to be treated via small openings, the attending surgeon usually has no direct view of the operating field. Direct view of the operating field is therefore replaced in such surgical interventions by a video imaging of the operating field, this usually ensuing with optical means integrated into an endoscope, a laparoscope or a catheter. In contrast to a direct view of the operating field, the viewing conditions in the video imaging are significantly limited.
An improvement of the view is achieved when, given minimally invasive interventions, the endoscope, the laparoscope or the catheter or, respectively, the respective optical system thereof is pivoted for image registration. Further, the view of the operating field can be improved by increasing the distance of the optical system from the subject to be imaged.
Another approach for expanding the field of vision of a physician is disclosed in U.S. Pat. No. 5,912,720 for the treatment of eye maladies wherein the superimposition of stored optical or angiographic images of a subject with one or more optical images of the subject acquired in real time ensues. The superimposition of the images is thereby based on a registration of anatomical landmarks, i.e. specific anatomical peculiarities of the registered tissue that are imaged in the images to be superimposed on one another and serve as a point of reference for the superimposition.
German OS 198 00 765 discloses a method and a system for generating image presentations of the surface of the inside wall of hollow bodies within the framework of an endoscopy examination, particularly of cavity organs and vessels. A number of individual frames of different regions of the inside wall of the hollow body to be examined are registered with the endoscope. The video image signals are digitized and, based on the digitized video image signals, a superimposition or combining of the individual frames to form an overall image is undertaken in an image processing device.
U.S. Pat. No. 5,776,050 discloses supplementing a video image acquired with an endoscope with topological image data of a 3D image data set stored in a computer to form an image expanded by the topological image data. The position of the endoscope in a coordinate system different from the coordinate system of the 3D image dataset is thereby determined with a navigation system. In order to produce a relationship between the two coordinate systems, and thus the 3D image dataset and the video image data, so that a superimposition of the video image data with the topological image data is possible, a registration procedure, for example on the basis of anatomical landmarks, is implemented. Subsequently, video images supplied by the endoscope, due to the continuous positional acquisition of the endoscope by the navigation system and the transformation rule between the coordinate system determined by the registration, can be continuously supplemented with topological data to form an expanded image that increases the field of vision.
A disadvantage of this procedure, however, is that the required registration procedure is time-consuming and susceptible to error and is therefore difficult to integrate into a routine execution during the course of a minimally invasive surgical intervention.
In the section xe2x80x9cBildrektifikationxe2x80x9d, methods for geometrical and radiometric correction of picture elements for the purpose of matching two images are described in the xe2x80x9cLexikon der Computergrafik und Bildverarbeitungxe2x80x9d by Iwainsky, A. and Wilhelmi W., Vieweg Verlagsgesellschaft, 1994, pages 31, 32. Two methods for geometrical correction are described, one directed to the calculation of two-dimensional correction polynomials of the order k and the other is directed to a perspective transformation method.
German OS 40 21 102 discloses a medical diagnostic installation having two integrated, imaging systems, one being an x-ray system and the other being an ultrasound system. With the assistance of position sensors, the surface or the spatial allocation of an ultrasound tomogram generated with the ultrasound system with respect to the x-ray image generated by the x-ray system is determined in order to be able to mix the ultrasound tomogram into the x-ray image. To this end, the x-ray system and the ultrasound system, however, must be arranged in a defined way relative to one another in order to be able to produce a relationship between the image data of the x-ray system and of the ultrasound system.
Moreover, German OS 197 46 092 discloses an x-ray registration device in the form of a C-arm x-ray apparatus for acquiring a 3D image dataset. The 3D image dataset is generated from a series of 2D projections that are acquired successively from different projection directions relative to a subject. In order to be able to acquire such a 3D image dataset from 2D projections, however, knowledge of the projection geometries is required, i.e. the exact knowledge of the position of the x-ray source and of the x-ray receiver as well as of the projection angle during each and every individual 2D projection. In order to determine these positions, the C-arm x-ray apparatus has transmission devices arranged at the x-ray source and at the x-ray receiver and has reception devices stationarily arranged relative thereto, for example in the form of ultrasound transmitters and receivers.
An object of the present invention is to provide a system and a method of the type initially described wherein a superimposition or fusion of, for example, image data non-evasively acquired from the inside of a subject and video image data can ensue for producing an image without a registration procedure.
According to the invention, this object is achieved in a system for generating an image that contains superimposed or fused image data, having a first system for acquiring an image dataset from a subject, a second system for the registration of a video image of the subject, means for determining the position of the first system in the acquisition of the image dataset and the position of the second system in the registration of the video image, a unit for determining the position of the image dataset acquired with the first system and the position of the video image acquired with the second system, a unit for superimposition or fusion of image data of the respective image datasets acquired with the first and second systems, and a unit for generating an image from the superimposed or fused image data. In the invention, therefore, the position of the first system in the acquisition of the image dataset as well as the position of the second system in the registration of a video image are identified in a reference coordinate system, so that the position of the first and second systems relative to one another is known. Due to the registration parameters that are likewise known, the position of the image dataset acquired with the first system can be determined on the basis of the known positions of the first system in the acquisition of an image dataset. Due to the registration parameters that are likewise known, the position of the image plane of the video image in the reference coordinate system can be determined on the basis of the known position of the second system in the registration of the video image. Therefore, image data of the image dataset acquired with the first system and the video image data can be superimposed on one another or fused with one another. In this way, for example in minimally invasive medical interventions, intra-operative images can be produced without a registration procedure for producing a spatial relationship between the image data of the image dataset acquired with the first system and the video image data, said intra-operative images containing video image data and, for example, non-invasively acquired image data from the inside of the subject. Video images are preferably produced that are supplemented by the non-invasively acquired image data of the image dataset produced with the first system, so that the possibly limited field division of a video image is expanded. The video image data are, for example, supplemented by image data from the image plane of the image dataset corresponding to the image plane of the video image. The presentation of the generated images can, for example, ensue as a picture-in-picture display.
In a preferred embodiment of the invention the unit for determining the positions of the first and of the second systems also can determine the position, i.e. the attitude and orientation, of the subject in space. In this way, for example given minimally invasive medical interventions, non-invasively, pre-operatively acquired image data of the subject also can be employed for superimposition or fusion with video image data registered during the minimally invasive surgical intervention. As a result of the acquisition of the position of the subject in the pre-operatively undertaken registration of an image dataset with the first system and as a result of the acquisition of the position of the subject during the implementation of the actual surgical intervention at the subject, modifications in the position of the subject can be taken into consideration and the image dataset can be brought into agreement with respect to its attitude in space with the modified attitude of the subject, or can be adapted to the modified attitude of the subject. Moreover, the determination of the position of the subject is also meaningful when the subject to be examined moves or is moved during the intervention between the acquisition of the image dataset with the first system and the registration of the video image. Accordingly, the subject is in a different attitude in the image dataset acquired with the first system than in the registration of the video image with the second means. By acquiring the movements of the subject, however, it is possiblexe2x80x94taking the movements of the subject into considerationxe2x80x94to bring the image dataset acquired with the first system into agreement with the new attitude of the subject, so that a video image registered given the new attitude of the subject can be unproblemmatically superimposed or fused with image data of the adapted image dataset.
In a version of the invention, the unit for determining the positions of the first and of the second systems is a navigation system. The navigation system includes an arrangement for non-contacting determination of the position of a subject. The navigation system can, for example, be a known optical navigation system, an electromagnetic navigation system, a navigation system based on ultrasound or of some other known navigation system that is known in and of itself.
In a preferred embodiment of the invention a 3D image dataset is acquired with the first system. The first system can be an x-ray system, preferably a C-arm x-ray system, or an ultrasound system, and the second system can be an endoscope, a laparoscope or a catheter. The advantage of employing a C-arm x-ray system or an ultrasound system for non-invasive acquisition of an image dataset is a relatively economical generation of the image datasets compared to the acquisition of datasets with an x-ray computed tomography apparatus or a magnetic resonants apparatus.
The above also is achieved in a method for generating an image that contains superimposed or fused image data, having the following method steps:
a) Acquiring an image dataset of a subject with a first system for acquiring image data;
b) Registration of a video image of the subject with a second system;
c) Determination of the position of the first system in the acquisition of the image dataset and the position of the second system in the registration of the video image;
d) Determining the position of the image dataset acquired with the first system and the position of the video image registered with the second system;
e) Superimposing or fusing image data of the first image dataset acquired with the first system and image data of the video image registered with the second system; and
f) Generating an image from the superimposed or fused image data.
In the inventive method, thus, as discussed in conjunction with the inventive system, the position of the image dataset acquired with the first system and the position of the video image registered with the second system relative to one another in a reference coordinate system are identified on the basis of the identified position of the first system in the acquisition of the image dataset and the position of the second system in the registration of the video image with respect to a reference coordinate system. Since, accordingly, the attitude of the image data of the image dataset relative to the attitude of the video image data is known, the image data can be superimposed on one another or be fused with one another. As a result, images can be generated that contain video image data and, for example, non-invasively acquired image data from the inside of a subject.
In a further version of the inventive method, the position of the subject also is identified; the identification of the position of the first and of the second systems as well as the determination of the position of the subject ensues with a navigation system. The employment of a know navigation system represents an especially practical form for determining the positions of the first and second system s as well as the position of the subject.